The present invention relates generally to image scaling methods. Two-dimensional (2D) spatial scaling is more specifically addressed. In particular, the method is well suited to the processing of video sequences.
Upscaling a grayscale or color image is useful to display the image with a spatial resolution higher than the resolution of the image signal as received, for example for displaying a PAL or NTSC TV signal in a HDTV format. The upscaling operation, however, often leads to artifacts typical of scaling aliased images.
Upscaled edges have staircase effects (or “jaggies”), which have unnatural motion when such edges are moving in a video sequence. This is caused by aliasing, i.e. the frequency content of the original image has been folded by the sampling applied when acquiring or transforming the signal and the folded frequencies are not at appropriate locations in the 2D spectrum after upscaling. The generation process of an aliased image with an aliased spectrum is illustrated in FIG. 1. Ideally, an image portion (patch) containing an edge should have a relatively sharp spectrum as shown in the bottom left portion of the figure. But oftentimes, the sampling rate fs of the image portion is not sufficient to ensure presence at the right spectral locations of the high-frequency components of the signal. Instead, these high-frequency components are folded and appear at other spectral locations as shown in the bottom right portion of FIG. 1, which corresponds to the jaggy aspect of the edge in the subsampled image.
A standard way of upscaling an image is to apply an interpolation filter. This is illustrated in FIGS. 2-4. Depending on the aliasing of the subsampled image, aliased spectral contents can be at various locations (arrows in the spectrum of FIG. 2). In general, the filter, whose spectrum is typically as shown in FIG. 3, is not able to properly recover a high-resolution image without leaving some amount of aliased spectrum (arrows in FIG. 4 showing the aliased spectrum of the upscaled image).
On the other hand, if the subsampled image having an aliased spectrum is upscaled using a filter whose spectrum (FIG. 5) is directionally selective, the upscaled image (FIG. 6) has a spectrum much closer to that of the original image. It does not have any more aliased contents, and it keeps a more important part of the original high-frequency content.
Directional interpolation methods have been proposed, for example in US 2009/0028464 A1. Similar (yet different) problems are addressed in U.S. Pat. No. 6,614,484 in the field of deinterlacing. An interpolation method consists in providing a set of directional interpolation filters, and performing the interpolation by choosing for each pixel an interpolator depending on the local image content. The underlying idea is that it is better to use a directional interpolation filter that is aligned with a contour whenever the current pixel is on a contour within the image.
Existing solutions for directional interpolation are usually based on very simple 2-tap filters. The metrics used to select a particular directional interpolation are usually simple gradient or correlation metrics. Such solutions are still prone to visual artifacts such as blurring of sharp edge patterns and detailed texture patterns in the image, ringing along edge contours, as well as jaggedness along edge contours.
There is thus a need for improved image processing methods in the field of directional interpolation or 2D scaling.